Probabilistic Algorithms, Integration, and Empirical Evaluation for Disambiguating Multiple Selections in Frustum-Based Pointing

There are a few fundamental pointing-based user interface techniques for performing selections in 3D environments. One of these techniques is ray pointing, which makes selections by determining intersections of objects along a single ray cast from the user. This technique is susceptible to inaccuracy of the user interface tracking technology and user noise (e.g., jittery hand, see actual viewing ray in Figure 1 (bottom) and how it misses the object when error is factored in). Another technique is the frustum-based (or equivalent cone-based) approach which casts a frustum from the user into the 3D scene. Selections are made by performing intersections of objects with the frustum. This technique resolves the imprecision associated with the ray pointing technique (see how the lightly shaded cone in Figure 1 (bottom) still intersects the object), but may produce multiple ambiguous selections as shown in Figure 2. This paper presents probabilistic selection algorithms and integration schemes that address some of the ambiguities associated with the frustum-based selection technique. The selection algorithms assign probabilities that the user has selected particular objects using a set of low-level 3D intersection-based selection techniques and the relationship of the objects in a hierarchical database. Final selections are generated by integating the outputs from each selection algorithm using one of several weighting schemes. We implemented the selection algoithms and weighting schemes within our distributed augmented reality (AR) and virtual reality (VR) architecture. Next, we performed several experiments to empirically evaluate the probabilistic selection techniques and integration schemes. Our results show that the combined selection and integration algorithms are effective at disambiguating multiple selections. This paper thoroughly describes the architecture, experiments, and results.